An Automatic Close Copy Speech Synthesis Tool for Large-Scale Speech Corpus Evaluation

The production of rich multilingual speech corpus resources on a large scale is a requirement for many linguistic, phonetic and technological tasks, in both research and application domains. It is also time-consuming and therefore expensive. In particular the human component in the resource creation process is prone to inconsistencies, a situation which has frequently been documented in studies of cross-transcriber consistency in manual time-aligned signal annotation. In the present case, corpora of three languages were to be evaluated and corrected: (1) Polish, a large automatically annotated and manually corrected single-speaker TTS unit-selection corpus in the BOSS Label File (BLF) format, (2) German and (3) English, the second and third being manually annotated multi-speaker story-telling learner corpora in Praat TextGrid format. A method is provided for supporting the evaluation and correction of time-aligned annotations for the three corpora by permitting a rapid audio screening of the annotations by an expert listener for the detection of perceptually conspicuous systematic or isolated errors in the annotations. The criterion for perceptual conspicuousness was provided by converting the annotation formats into the interface format required by the MBROLA speech synthesiser. The audio screening procedure is complementary to other methods of corpus evaluation and does not replace them. Conceptually the ACCS synthesis tool is intended as an extension of the BLARK toolkit for speech corpora. 1. Efficient quality control of richly annotated corpora The production of rich multilingual speech corpus resources on a large scale is a requirement for many linguistic, phonetic and technological tasks, in both research and application domains. It is also time-consuming and therefore expensive. In particular the human component in the resource creation process is prone to inconsistencies, a situation which has frequently been documented in studies of cross-transcriber consistency in manual time-aligned signal annotation, particularly with prosody (Grice 2006; Gibbon et al. 1997; Gibbon et al. 2000). In the present case, corpora of three languages were to be evaluated and corrected: (1) Polish, a large automatically annotated and manually corrected single-speaker TTS unit-selection corpus in the BOSS Label File (BLF) format (Demenko et al. 2006), (2) German and (3) English, the second and third being manually annotated multi-speaker story-telling learner corpora in Praat TextGrid format (Boersma 2001; Gut et al. 2004). The first general goal is to provide a method for supporting the evaluation and correction of time-aligned annotations for the three corpora by permitting rapid audio screening of the annotations by an expert listener, who detects perceptually conspicuous systematic or isolated errors in the annotations. The criterion for perceptual conspicuousness is provided by converting the annotation formats into the interface format required by a suitable speech synthesizer, in this case the PHO format required by the MBROLA synthesizer (Dutoit & al. 1996). The audio screening procedure is complementary to other methods of corpus evaluation and does not replace them. Functionally, the ACCS synthesis tool is intended as an addition to the BLARK (Krauwer 2005) toolkit for speech corpora. A second general goal is a practical one: the method is also intended for use with corpora for less-resourced languages and for use in areas with very basic infrastructures. The method should therefore not only be of good quality and well-defined, but at the same time straightforward and as far as possible not dependent on complex cutting edge tools, expensive software packages, or the internet. Otherwise, usability by development teams working with sub-optimal infrastructures in an under-resourced languages paradigm is not assured. The focus in the present application is on segmental annotation evaluation, but the pitch pattern of the original speech signals was also extracted and mapped into the synthesiser interface in order to provide as natural a re-synthesis as possible for the segmental evaluation. Prosodic annotation is not evaluated. The issues addressed are: 1. Consistency of labels used with defined label set (e.g. phoneme or phone set). 2. Correct time-stamp assignment (e.g. segment duration). 3. Correct label selection from the relevant inventory. The first of these problems is a „syntactic‟ issue which can be dealt with automatically, given a specified inventory of labels. The second and third are „semantic‟ issues which require an element of subjective assessment, in that mapping of the annotation to the speech signals is involved. This assessment can be cross-transcriber